Nonorthogonality in Ill–Conditioned Systems

A ridge version of an ill–conditioned system is long held to “act more like an orthogonal system” than the system itself. Surrogate models exist also having smaller expected mean square than OLS, and surrogate solutions are seen to have uniformly smaller residual sums of squares than ridge. Ridge and surrogate solutions are compared on several hallmarks of orthogonality, to include conditioning of dispersion matrices, variance inflation factors, isotropic variances, and a sphericity index for probability contours and for ellipsoids of concentration of the estimators. On these criteria, ridge solutions are shown to exhibit erratic behavior, diverging from orthogonality as the ridge scalar evolves; whereas surrogate solutions increasingly resemble those from orthogonal systems. These limitations trace to a failure of ridge to address adequately the ill-conditioning intrinsic to the original system. Computations are simplified by invariance to those appropriate for canonical models based on singular decompositions. Case studies illustrate the central issues.